Human Machine Interfaces (HMIs) for intelligent machines and robots is an active research topic with literally hundreds of papers on the subject. An overview of the field including related references is provided in M. McDonald, “Active Research Topics in Human Machine Interfaces”, Sandia Report, SAND2000-2779, December 2000, which is herein incorporated by reference in its entirety. A significant branch of the Human-Computer Interface (HCI) design community outside HMI is focused toward studying user needs within an ethnographic context as described in texts including for example, “Bringing Design to Software” by Terry Winograd, ACM Press, New York, N.Y., 1996, ISBN: 0201854910, which is herein incorporated by reference in its entirety. However, relatively little research or development has been done to apply HCI user studies and ethnographic techniques to Human Machine Interfaces.
After analyzing several human machine interfaces on typical existing robot systems against a variety of user needs, it can be seen that most human machine interfaces lack a strong concept of user tasks and do not adequately draw on notions that are familiar to the community of users of robotic equipment for task handing such as robot handlers in the bomb squad user community and the like.
Within the bomb squad and crime investigation communities, for example, user processes or tasks are typically defined in terms of overall task requirements. For example, bomb squad responders often describe situations and develop conceptual solutions to their problems in terms of sketches prior to performing actual bomb retrieval and disposal operations. Homicide investigators likewise use sketches at crime scenes to generate and test crime theories so that they can both better organize the evidence that they collect and explore theories that help them better identify which evidence is worth collecting. Finally, these sketches of task plans are often used to communicate and develop best practices for subsequent training activities.
Modern HMIs do not accommodate typical practices such as task orientation and sketch generation, and often the HMI imposes a layer of complexity between actually performing the task and controlling robotic, machine and/or virtual object motion.
It would be desirable therefore in the art for a method and apparatus for allowing robotic, machine and/or virtual object motion to be mutually and directly controlled by practices already employed by those charged with accomplishing tasks using virtual environments, robots or machines.